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Abstract 

Recent studies in higher education suggest that nontraditional students view their 
education in the same light as any other form of market exchange. Subsequently, they 
demand high service quality that reflects convenience, flexibility, and value. Responding 
to the needs and wants of these students, nontraditional colleges have adopted research 
strategies that take into account both student expectations as well as their perception of 
satisfaction to assess service quality at their institutions. As one of the model adult 
learner focused institutions, Empire State College used Noel-Levitz Adult Learner 
Inventory in Fall 2002 to measure adult students’ satisfaction with a wide range of 
college experiences. The paper investigates response data from the survey using 
importance- satisfaction gap analysis, factor analysis and quadrant analysis to study 
important dimensions of service quality for assessment and quality improvement 
initiatives. Results suggest that nontraditional students value responsiveness and 
relationships as key dimensions in evaluating the teaching-learning and service delivery 
processes. 



Background and Literature Review 

Higher education in the United States has witnessed a paradigm shift over the past 
few years. As Michael, et al. (1995, p.22) observe, this shift is subtle and steady and 
“entails, among other things, a shift from pre-employment, youth-based education to a 
lifelong, adult-oriented education.” This shift is also a reflection of changing student 
demographics, which shows adult nontraditional learners as the new undergraduate 
majority (CAEL, 2003). Recent data from National Center for Educational Statistics 
(NCES) suggest that 73% of the undergraduate students in the United States have one or 
more nontraditional characteristics (2002). The demographic profile of the contemporary 
undergraduate students in the United States reveal that 44% of these students are mature 
adults over the age of 25, around 54% hold down full-time jobs or careers (Wright & 
O’Neill, 2002), and 51% are financially independent of parents (NCES, 2002). The 
changing profile of contemporary college student has encouraged traditional colleges to 
develop programs and practices that suit adult nontraditional learners. This has presented 
a new challenge to higher education institutions whose mission is to primarily serve 
nontraditional students. Given the new competition from the traditional institutions, 
nontraditional institutions are trying to address the different needs and wants of their 
nontraditional students. Recent research suggests that compared with young traditional 
students, contemporary adult nontraditional students have significantly different 
expectations of service quality. Nontraditional students “view their education in the same 
light as any other form of exchange,” and subsequently, demand high service quality that 
reflects convenience, flexibility, and value (Wright & O’Neil, 2002; Haworth & Conrad, 
1996; West, 1999). As a result, there has been a new shift in the research agenda within 
higher education institutions, and as Wright & O’Neil suggest, the focus has shifted 
“from quality of education to quality of service” (2002). The shift in the research focus 
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has been more pronounced among the institutions whose mission is to serve 
nontraditional students. Nontraditional institutions often find it difficult to find 
standardized instruments for measuring and benchmarking service quality. Recognizing 
the need for such standardized assessment tools for institutions serving adult learners, the 
Adult Learner Inventory was developed by Noel-Levitz and Council for Adult and 
Experiential Learning (CAEL) to measure adult students’ satisfaction with a wide range 
of college experiences. The Adult Learner Inventory views students as “consumers” who 
have a choice about investing in educational opportunities. It recognizes that adult 
students have “definite expectations about what they want from their campus experience. 
From this perspective, satisfaction with college occurs when an expectation is met or 
exceeded by an institution” (Noel-Levitz, 2003). By using an importance- satisfaction 
scale, the inventory allows for an importance-satisfaction performance gap analysis, 
which has been used to measure service quality in the present study. 

The importance-satisfaction performance gap analysis is based on the 
SERVQUAL model developed and improvised by Parasuraman et al., to study 
performance gap or “disconfirmation” between consumer experience and expectations 
(1985, 1988). Although, the SERVQUAL model has been adapted and successfully used 
in educational research (Ford et al., 1999), it has been criticized for its rigid dimensions 
which do not have universal applicability (Cronin & Taylor, 1992; Carman, 1990); its 
inability to account for the changing nature of consumer expectations over time, and its 
undue focus on expectations (Chapman, 1979). As an alternative to the SERVQUAL 
model with performance-expectation gaps, researchers have suggested the use of 
importance-satisfaction performance gap analysis to evaluate service quality in higher 
education (Ford et al., 1999; Martilla & James, 1977; Wright & O’Neill, 2002). 

In Fall 2002, 25 adult learner focused institutions participated in a pilot project to 
administer the Adult Learner Inventory to more than 13,000 of their students. As one of 
the model CAEL adult learner focused institutions, Empire State College was invited to 
participate in the pilot project. This paper investigates response data from the pilot project 
to study important dimensions of service quality for assessment, marketing, and quality 
improvement initiatives. 



Method 



Subjects and Procedure 

The Adult Learner Inventory was administered by the college electronically via 
the Internet between October 10, 2002 and November 29, 2002. The survey link and the 
password were e-mailed to 4,194 students who were enrolled at the college at the time of 
the survey administration and held a valid e-mail address. A total of 1,954 responses 
were received by the college, yielding an overall response rate of 46.6%. 



Sur\’ey Instrument 

As noted above, the Noel-Levitz Adult Learner Inventory was used as the survey 
instrument. The inventory had seven-point Likert-type scales to collect both importance 
and satisfaction ratings on 54 attributes. It also included questions on student 
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demographics and two summary items related to student overall satisfaction with the 
institution. The inventory construction was based on seven principles, that were identified 
by CAEL as effective principles for serving adult students: outreach, life and career 
planning, financing, assessment of learning outcomes, teaching-learning process, student 
support systems, and technology. 

Results 



Demographics 

A broad section of gender, age, and ethnicity were represented in the survey. 
Sixty-six percent of the respondents were women, 92.2% of the respondents were 25 or 
older, 80.03% of the respondents were White/Caucasian, 8.69% were Black/ African- 
American, 4.98% were Hispanic, and 1.43% were Asian. Further analysis of the 
responses to the demographic questions on the survey revealed that 65.65% of the 
respondents were married or had a domestic partner, 61.83% had dependents, and 
73.32% had full-time employment. 

Factor analysis and survey reliability 

The results showed that the Adult Learner Inventory performed well in terms of 
both reliability and validity. A factor analysis was conducted to validate the performance 
gap scores (importance scores-satisfaction scores). Using principal component method 
and varimax rotation, the factor analysis identified nine dimensions of service quality: 
mentoring/advising, financial access, adult focused learning, flexible curriculum, 
technology, interactive learning, convenient programs/procedures, information access, 
and instructor responsiveness. These nine factors were able to explain 65.34% of the total 
variance. Table 1 displays the total variance explained by the nine factors. 

Tests of internal consistency across different dimensions showed that overall, the 
Adult Learner Inventory was a fairly reliable tool to measure service quality dimensions 
among the nontraditional college students. According to Nunnally (1978), reliability 
coefficients greater than .70 are standard for adequate reliability of questionnaires and 
protocols. The chronbach alpha score for the nine service quality dimensions identified 
by factor analysis ranged between 0.72 and 0.89 establishing the overall reliability of the 
survey instrument. Table 2 displays factor means and reliability estimates. 
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Table 1. Total Variance Explained by the Model 





Initial 

Eigenvalues 




Extraction Sums of 
Squared Loadings 




Rotation Sums of 
Squared Loadings 




% of Cumulative 


% of 


Cumulative 




% of 


Cumulative 


Factor Total 


Variance % 


Total 


Variance 


% 


Total 


Variance 


% 


1 21.92 


40.58 40.58 


21.92 


40.58 


40.58 


4.83 


8.95 


8.95 


2 2.78 


5.15 45.73 


2.78 


5.15 


45.73 


4.81 


8.91 


17.85 


3 2.19 


4.06 49.79 


2.19 


4.06 


49.79 


4.53 


8.39 


26.24 


4 1.81 


3.35 53.14 


1.81 


3.35 


53.14 


4.42 


8.19 


34.43 


5 1.78 


3.29 56.43 


1.78 


3.29 


56.43 


4.05 


7.50 


41.94 


6 1.40 


2.59 59.02 


1.40 


2.59 


59.02 


3.84 


7.10 


49.04 


7 1.32 


2.44 61.46 


1.32 


2.44 


61.46 


3.52 


6.52 


55.55 


8 1.23 


2.29 63.75 


1.23 


2.29 


63.75 


2.70 


4.99 


60.55 


9 1.13 


2.09 65.83 


1.13 


2.09 


65.83 


2.59 


4.80 


65.34 


Table 2. Factor Means and Reliability Estimates 


Factor 


Factor Importance Satisfaction Performance 


Standardized 


Code 


Name 


Mean 


Mean 


Gap 




Alpha 


Alpha 




Mentoring/ 














Factor 1 


Advising 


6.4 


5.6 


0.8 




0.89 


0.89 




Financial 














Factor 2 


Access 


6.1 


5.7 


0.4 




0.85 


0.85 




Adult Focused 














Factor 3 


Learning 


6.1 


5.5 


0.6 




0.88 


0.88 




Flexible 














Factor 4 


Curriculum 


6.3 


5.7 


0.6 




0.86 


0.86 


Factor 5 


Technology 


6.2 


5.7 


0.5 




0.71 


0.72 




Interactive 














Factor 6 


Learning 


5.7 


5.4 


0.3 




0.76 


0.76 


Convenient Programs/ 














Factor 7 


Procedures 


6.3 


5.7 


0.6 




0.83 


0.83 


Factor 8 Information Access 


6.5 


6.0 


0.5 




0.83 


0.83 




Instructor 














Factor 9 


Responsiveness 


6.2 


5.4 


0.8 




0.79 


0.80 



Quadrant analysis and evaluation of gap scores 

Once the factors were isolated, a quadrant analysis was used to find areas that 
needed special attention for quality improvement, marketing, and effective planning. The 
quadrant analysis was based on a scatter plot of satisfaction and performance means of 
the nine factors identified by factor analysis of the gap scores. Figure 1 presents the 
results of the quadrant analysis. Quadrant A represents high importance and high 
satisfaction; Quadrant B represents high importance and low satisfaction; Quadrant C 
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represents low importance and low satisfaction; and Quadrant D represents low 
importance and high satisfaction. 

Figure 1. Matrix for Prioritizing Action 



Student Satisfaction and Importance Grid by Service Quality Factors 
Overall Importance Mean=6.21 and Overall Satisfaction Mean=5.66 



6.40 



6.20 



Quadrant B 
Concentrate Here 



| Mentoring & Advising 
I Instructor Responsiveness 



+ 



Convenient Programs & 

Procedures/Flexible 

Curriculum 



Information Access 



Quadrant A 

Keep up the good work 



d) 

2 6.00 

C 

ra 

C 5.80 

O 

Q. 

E 5.60 



5.20 



5.00 



Quadrant C 
Low Priority 



X 



Technology 



Interactive Learning 



Financial Access 



Adult Focused Learning 



Quadrant D 

Possibly too many resources 



5.00 



5.20 



5.40 



5.60 5.80 

Satisfaction 



6.00 



6.20 



♦ Mentoring/ Advising 

Flexible Curriculum 
Information Access 



Financial Access 
Interactive Learning 
Instructor Responsiveness 



▲ Adult Focused Learning 

+ Convenient Programs/Procedui)e 

X Technology 



The four quadrants present a matrix for prioritizing action (Noel-Levitz, 2003). 

As shown in Fig. 1, Quadrant A includes convenient programs/procedures, flexible 
curriculum, and information access. These areas represent areas of strength that could be 
used for marketing and public relations. These areas need to be monitored, but do not 
need immediate resources (Westfall, 2003). Factors in Quadrant B, on the other hand, 
require immediate attention for quality improvement efforts as they represent areas of 
high disconfirmation. Instructor responsiveness and mentoring/advising were the two 
factors located in Quadrant B . Items related to these factors showed large performance 
gaps. Further analyses of the gap scores on mentoring/advising items revealed that 
timeliness of help in developing a study plan, receiving help to make decisions about 
courses and programs, and timely response to request for information were key attributes 
that needed immediate attention. The gap scores on instructor responsiveness and 
mentoring/academic advising items are presented in Table 3 and Table 4. Among the 
items on instructor responsiveness, timely feedback on academic progress, instructors 
taking time to discuss educational goals with students, and instructors’ involvement in 
assessing student learning received the lowest satisfaction ratings. Items related to 
interactive learning and adult focused learning fell in Quadrant C, suggesting low priority 
items not needing immediate attention. Items related to technology and financial access 
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fell in Quadrant D suggesting a possibility of too many resources being allocated to these 
areas. Looking at the priority matrix, it appears that some of the resources from 
technology and financial services may well be diverted to provide training to faculty on 
student advising and instructional planning and delivery. 

Table 3. Mentoring and Academic Advising 





Importance 


Satisfaction 




Attribute 


Mean 


S.D. 


Mean 


S.D. 


Gap 


This institution provides students with the help they need to 
develop a plan of study before enrolling 


6.3 


1.0 


5.2 


1.8 


1.1 


I receive the help I need to make decisions about courses and 
programs that interest me 


6.5 


0.8 


5.6 


1.6 


0.9 


I receive timely responses to my requests for information 


6.5 


0.7 


5.6 


1.5 


0.9 


My advisor is knowledgeable about requirements for 
courses and programs of interest to me 


6.6 


0.7 


5.8 


1.6 


0.8 


This institution periodically assesses my skill level to guide 
my learning experiences 


5.7 


1.3 


4.9 


1.5 


0.8 


Staff here are available to help me solve unique problems I 
encounter 


6.4 


0.9 


5.6 


1.5 


0.7 


Mentors are available to guide my career and life goals 


6.4 


1.1 


5.7 


1.6 


0.7 


My advisor is available either by phone, fax, e-mail, or online 
when I need help 


6.6 


0.7 


6.0 


1.4 


0.6 



Table 4. Instructor Responsiveness 





Importance 


Satisfaction 




Attribute 


Mean 


S.D. 


Mean 


S.D. 


Gap 


My instructors provide timely feedback 
about my academic progress 


6.5 


0.8 


5.5 


1.6 


1.1 


My instructors take the time to discuss 
my educational goals with me 


6.3 


1.0 


5.4 


1.7 


0.9 


My instructors respect student opinions and 
ideas that differ from their own 


6.4 


0.9 


6.0 


1.5 


0.4 


My instructors involve me in 
assessing my own learning 


6.0 


1.2 


5.5 


1.3 


0.5 



Discussion 

Traditional student satisfaction surveys measuring only the student perception of 
satisfaction with service quality have limited utility for nontraditional institutions, as they 
are unable to address the disconfirmation between student importance and satisfaction 
ratings. Results from the present study show that, compared with an analysis based purely 
on satisfaction scores, importance-satisfaction performance gap analysis provide a more 
accurate measure of service quality. Across all dimensions identified in this study, 
satisfaction scores had smaller means and larger standard deviations than the importance 
scores. The high scores on importance items suggest that like other consumers engaged in 
a commercial exchange activity, nontraditional students have high expectations of service 
quality from their educational institutions (Boulding, 1993). In addition, lower standard 
deviations on importance items suggest that the general agreement among the 
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respondents was more on importance items and less on satisfaction items (Westfall, 
2003). 

Compared with national averages, Empire State College students showed higher 
satisfaction on all dimensions. Despite their high satisfaction ratings, Empire State 
College students showed a general disconfirmation with service quality items as the 
survey revealed positive value of performance gap (importance score-satisfaction score) 
scores on all 54 attributes. This further confirms that satisfaction scores alone cannot 
present a holistic assessment of service quality. The focus on disconfirmation between 
importance and satisfaction provides gap analysis an objective context to evaluate service 
quality for quality improvement, marketing, and resource allocation. Moreover, in terms 
of prioritizing action, gap analysis provides a focused action plan for resource allocation 
by pinpointing low priority and high priority areas, thereby facilitating diversion of 
scarce resources to areas where improvements can make strongest impact on customer 
satisfaction and retention. 

Results from quadrant analysis indicate that attributes related to “instructor 
responsiveness” and “mentoring/advising” had the largest performance gaps with high 
importance scores and low satisfaction scores. Structural issues like financial access and 
technology were important to students. Nevertheless, there was less “disconfirmation” on 
items related to these dimensions. Results from quadrant analysis not only underscore the 
relationship aspect of the teaching-learning process, they also validate the assertion that 
adult nontraditional students value relationships that are friendly partnerships, “much 
like the partnership formed between banks and supermarkets and the modern-day 
consumer” (Wright & O’Neill, 2002; Levine & Cureton, 1998). 
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